Skip to content

feat(toolchain): make opsm.toml [runtime] the single source of truth - #67

Merged
hyperpolymath merged 2 commits into
mainfrom
claude/odds-sods-pm-goals-xktszo
Jul 9, 2026
Merged

feat(toolchain): make opsm.toml [runtime] the single source of truth#67
hyperpolymath merged 2 commits into
mainfrom
claude/odds-sods-pm-goals-xktszo

Conversation

@hyperpolymath

@hyperpolymath hyperpolymath commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Problem

OPSM had no single source of truth for its own toolchain versions — five sources disagreed:

Source Erlang Elixir Notes
.tool-versions (what actually builds it) 28.3.1 1.19.5-otp-28 + vestigial nodejs 25.6.1
opsm.toml [runtime] (read by opsm runtime install) 26.2.0 1.16.0 stale
.gitlab-ci.yml elixir:latest rust:latest non-reproducible
e2e.yml + nif-build.yml 27.0 1.17 hardcoded
trust-pipeline-e2e.yml 26.2 1.16.0 hardcoded

Plus: Nickel heavily used (runtime/**/*.ncl) but pinned nowhere, Idris2 (a real build dep — src/abi/*.idr) unpinned, Rust floating on stable.

Fix

opsm.toml [runtime] is now canonical (dogfooding — opsm runtime install already consumes it). Everything else derives from it or is drift-checked against it:

opsm.toml [runtime]  ── just toolchain-sync ──►  .tool-versions ──► asdf/mise + setup-beam (version-file)
        └─ just toolchain-check (Mustfile-gated) ──► verifies .tool-versions + GitLab image pins
  • opsm.toml: corrected stale pins (erlang → 28.3.1, elixir → 1.19.5-otp-28); completed the set (rust 1.97.0, nickel 1.16.0, idris2 0.8.0); dropped the vestigial nodejs pin (no package.json, nothing invokes node — Deno is the sole JS runtime)
  • scripts/toolchain.sh (new, POSIX sh): sync regenerates .tool-versions; check fails loudly on any drift, incl. GitLab image tags. Wired as just toolchain-sync / just toolchain-check + Mustfile check
  • .tool-versions: now generated, with idris2 (no asdf plugin — pack/Chez route) and rust (asdf-global, cargo-audit conflict) excluded via documented skip-list. zig/deno/nickel provision via the estate's asdf-tool-plugins
  • GitLab CI: elixir:1.19.5-otp-28 + rust:1.97 (tags verified on Docker Hub); also fixed the invalid // comment on line 2 that broke YAML parsing. Scanner images (trivy/semgrep/trufflehog) deliberately float — fresh detection DBs beat cosmetic pinning
  • GitHub workflows: setup-beam now reads .tool-versions (version-file + version-type: strict) in e2e, nif-build, and trust-pipeline-e2e; rust steps pin toolchain: 1.97.0 (nif-build, echidna-validation)
  • parse_runtime_section/1 (manager.ex): hardened — comment lines containing = were mis-parsed as pins and inline comments corrupted versions; tests added
  • docs/TOOLCHAIN.adoc (new): derivation chain, per-tool provisioner table, the honest bootstrap story (OPSM is an Elixir escript — it cannot provision the BEAM for itself; asdf/mise/Guix sit underneath), and the Idris2 route (pack + Chez + GMP today; guix.scm is still a stub, documented as TODO rather than overclaimed)

Verified

  • sh scripts/toolchain.sh sync && sh scripts/toolchain.sh check → green
  • Drift injections (.tool-versions version mutated; GitLab image reverted to :latest) → both exit 1
  • All 5 edited YAML files parse cleanly
  • Elixir parser tests not run locally (no BEAM in this container) — CI runs them; watching this PR for fallout, especially trust-pipeline-e2e moving 26.2 → 28.3.1 (accepted risk per owner decision)

Known follow-ups (out of scope here)

  • guix.scm is a stub (source #f) — growing it into a real manifest is recorded in STATE.a2ml next-actions
  • Handoff sections B/C (service deps wiring, solver stack) — separate work

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C

Generated by Claude Code

claude added 2 commits July 9, 2026 23:00
@claude
feat(toolchain): make opsm.toml [runtime] the single source of truth
58079c0 
Five sources disagreed on OPSM's own toolchain versions (.tool-versions,
opsm.toml, .gitlab-ci.yml on :latest, and two different hardcoded sets
across GitHub workflows). This makes opsm.toml [runtime] canonical and
derives or drift-checks everything else.

- opsm.toml [runtime]: correct stale pins (erlang 26.2.0 -> 28.3.1,
  elixir 1.16.0 -> 1.19.5-otp-28); complete the set with rust 1.97.0,
  nickel 1.16.0 (previously unpinned anywhere), idris2 0.8.0; drop the
  vestigial nodejs pin (nothing uses Node; Deno is the sole JS runtime)
- scripts/toolchain.sh: sync (generate .tool-versions from opsm.toml)
  and check (fail on drift in .tool-versions or GitLab image pins);
  wired as just toolchain-sync / toolchain-check and gated in Mustfile
- .tool-versions: now generated; idris2 (no asdf plugin; pack/Chez route)
  and rust (asdf-global, cargo-audit conflict) excluded with comments
- .gitlab-ci.yml: pin elixir:1.19.5-otp-28 and rust:1.97 off :latest
  (tags verified on Docker Hub); fix invalid '//' comment on line 2;
  scanner images deliberately float for fresh detection DBs
- GitHub workflows: setup-beam now reads .tool-versions (version-file,
  strict) instead of three disagreeing hardcoded version sets; rust
  steps pin toolchain 1.97.0
- manager.ex parse_runtime_section/1: skip comment lines and strip
  inline comments (previously mis-parsed as pins); tests added
- docs/TOOLCHAIN.adoc: derivation chain, per-tool provisioners (incl.
  estate asdf-tool-plugins for zig/deno/nickel), the bootstrap layering
  under the escript, and the honest Idris2 story (pack today, guix.scm
  is still a stub)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
@claude
fix(elixir-1.19): resolve type-checker warnings surfaced by OTP 28 CI…
c002681 
… alignment

Moving CI from Elixir 1.17 to the canonical 1.19.5-otp-28 surfaced 9
warnings the older compiler never flagged; the e2e workflows compile
with --warnings-as-errors, so these were hard failures. Two were real
bugs, the rest dead code the new type checker proved unreachable.

Real bug fixes (package/cleanup.ex):
- cleanup/2: 'actions = actions ++ ...' rebinds inside the if block,
  which does not escape Elixir scoping — config/data/cache removal
  actions were silently dropped from the returned list
- remove_systemd_units/1: the for comprehension discarded its results,
  so the function always returned [] and daemon-reload never fired;
  now collects action tuples via comprehension filter

Dead-clause removals (callees verified infallible; a future callee
change fails loudly via MatchError/type check):
- trust/pipeline.ex: {:error, _} on Slsa.Provenance.generate/2 and on
  Oikos.analyze_package/4 (deliberately falls back to heuristic score)
- wiring.ex: same Oikos dead clause in run_sustainability_check/2
- package/installer.ex: {:error, _} on Pipeline.verify/2 (failed checks
  arrive inside {:ok, results}; service failures raise into rescue)
- runtime/source_builder.ex: :both strategy called a prebuilt_available?
  stub that always returned false — call source_install directly and
  drop the stub
- cli.ex: {:error, _} on Wiring.run_audit/2 (always {:ok, _})

Clause grouping (cli.ex): moved interleaved helpers (resolve_installed_version,
find_tui_binary, show_dependencies_*, collect_transitive_deps,
do_install_*) below the run/1 group so all 47 clauses are contiguous.
Pure reorder — content-identical lines, verified by sorted diff.

Verified locally on OTP 28.3.1 / Elixir 1.19.5 (builds.hex.pm):
mix compile --warnings-as-errors passes; 805 tests, 1 failure that is
a sandbox-proxy artifact (api.github.com 401 in a no-network test),
not reproducible on GitHub runners.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
@hyperpolymath Claude

hyperpolymath commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

CI status notes (after c002681):

  1. The two E2E failures were Elixir 1.19 fallout, now fixed. Moving CI from Elixir 1.17 to the canonical 1.19.5-otp-28 surfaced 9 warnings under --warnings-as-errors that the older compiler never flagged. Two were real bugs, not type noise:

    • cleanup/2 dropped config/data/cache removal actions from its returned list (actions = actions ++ … rebound inside an if block, which doesn't escape Elixir scoping)
    • remove_systemd_units/1 always returned [] and never triggered daemon-reload (the for comprehension discarded its results)

    The rest were provably dead {:error, …} clauses on infallible callees, plus run/1 clause-grouping. Verified locally on OTP 28.3.1 / Elixir 1.19.5: mix compile --warnings-as-errors passes; 805 tests with 1 failure that is a sandbox-proxy artifact (a "no-network" test reaching api.github.com through the session proxy and getting 401), not reproducible on GitHub runners.

  2. governance / Language / package anti-pattern policy is pre-existing, not from this PR. It fails identically on main at the base commit (run #94, head f91eb34) — 21 grandfathered .res files (cli/, lib/ocaml/, opsm_mobile/src/) from the estate ReScript→AffineScript migration. Declaring .hypatia-ignore exemptions is an owner-approval decision per estate policy, so this PR flags it rather than touching it.

Generated by Claude Code

@hyperpolymath
hyperpolymath marked this pull request as ready for review July 9, 2026 23:33
@hyperpolymath
hyperpolymath merged commit 6933186 into main Jul 9, 2026
26 of 28 checks passed
@hyperpolymath
hyperpolymath deleted the claude/odds-sods-pm-goals-xktszo branch July 9, 2026 23:33
@hyperpolymath hyperpolymath mentioned this pull request Jul 9, 2026
Merged
hyperpolymath added a commit that referenced this pull request Jul 9, 2026
@hyperpolymath @claude
fix(e2e): honor the live_download exclusion; fix runtime plugin searc…
6df4e8d 
…h path (#68)

Follow-up to #67. The e2e workflow has been red on `main` for weeks —
first at compile (`--warnings-as-errors`), which #67 fixed; with compile
green, the `runtime-api` job then failed on `{:error, {:no_plugin,
"zig"}}` from tests that were never meant to run there. Two pre-existing
defects, both fixed at source:

## 1. Tag taxonomy — download tests ran in PR CI despite the exclusion

The live-download tests carried **both** `@tag :external_api` and `@tag
:live_download`. ExUnit's `--include` re-includes tests dropped by
`--exclude`, so

```
mix test --include external_api --exclude live_download
```

ran the ~10–50MB download tests that `e2e.yml`'s own header reserves for
manual dispatch. They now carry only `:live_download` (run manually via
`--include live_download`), matching the taxonomy defined at the top of
the test file.

## 2. Plugin search path — `Manager.install` couldn't resolve plugins
from a checkout

`@plugin_search_dirs` was a **module attribute**, so
`:code.priv_dir(:opsm)` was frozen at *compile* time, pointing inside
whatever `_build` compiled the module — its own comment said "resolved
at runtime", an intent that never held. Result: `runtime/core/*.ncl`
plugins were unreachable from any repo checkout (this also breaks the
`opsm runtime install` dogfooding path that reads `opsm.toml
[runtime]`).

Search dirs are now computed per call, with repo-checkout fallbacks
(`cwd/../runtime/core` under mix, `cwd/runtime/core` as escript) and a
fail-safe guard on `:code.priv_dir/1`. `find_plugin_ncl/1` is `@doc
false` public with unit tests covering checkout resolution.

Also hardened: `System.cmd("nickel", …)` raises `ErlangError :enoent`
when nickel isn't installed — now returns `{:error,
{:nickel_not_installed, tool}}`.

## Verified (OTP 28.3.1 / Elixir 1.19.5)

- `mix compile --warnings-as-errors` — green
- `mix test test/opsm/runtime/manager_test.exs` — 17/17 (2 new
`find_plugin_ncl` tests)
- The exact `runtime-api` CI command — **9 tests, 0 failures, 6
excluded**, live against ziglang.org / go.dev / nodejs.org; both
previously-failing tests correctly excluded

Note: the branch was restarted from `main` after #67's squash-merge (the
old remote branch was auto-deleted on merge), so this PR contains only
the follow-up commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C)_

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hyperpolymath @claude